Metrics For Product Managers

I’ve recently had a chat with my team on metrics - one of the key topics a PM needs to be well-versed in. Metrics are at the core of helping you define what you’re truly after. Having a deep understanding of what you are measuring is the only way to make sure that in the long-run, you’re able to build a product that truly satisfies customer needs. Consider this post a basic intro in just that - getting a grasp on quantifiable measures of the performance of your product.

Top-level metric categories #

Any product typically has a spectrum of metrics that need to be measured. While those can vary depending on the field, target audience and type of offering, the overarching theme is that those should tell you how well your product is solving the problem it set forth to solve. Good metrics allow you to better understand customer needs and identify future opportunities, therefore it’s key that you spend some amount of time defining them.

Sachin Rekhi very eloquently outlined the types of metrics one would need:

• Acquisition - tracking how you acquire new users. Answers the question: how are you doing in terms of getting new users for your feature/product?
• Engagement - tracking how your users are leveraging your product functionality. It should capture the various ways in which your users are interacting with what you delivered, from activation (first meaningful contact with your product) to retention (coming back to your product).
• Monetization - tracking how you’re making money. Answers the question: how effective are you in generating revenue for your product?

Each category can have specific measures, but would ultimately roll-up to the higher-level umbrella “bucket”.

Be thoughtful in defining your metrics #

I am firm believer in actionable, accessible and auditable metrics. These properties were defined by Eric Ries in his Lean Startup work - they ring true for any product. Let’s quickly dissect each of them:

• Actionable metrics help you make decisions based on them. It doesn’t need to be one metric in isolation (it can be an aggregate of many), but ultimately looking at them you should be able to draw insights as to what your next steps are. If you can’t, you’re looking at vanity metrics - metrics that may make you feel good in the short-term, but will not positively influence your product decision-making process.
• Accessible metrics are understood by your entire team and don’t require a PhD in data science. If you can explain what you’re looking at to your peers that don’t necessarily work on the segment you’re analyzing, and they can interpret the data or at least get a solid grasp of what you’re trying to do, you’re doing well.
• Auditable metrics allow anyone on your team to cross-check the data and have a reliable way to see whether it’s correct or not. You need to be able to look at metrics critically and question every single value to make sure that you’re getting reliable information. Your team should be able to do the same thing.

From above, you can see a theme emerge - you might not need thirty different metrics. You might need just three, but those should be very precise and steer your product. Often I hear PMs try to use the “shotgun approach” - define a large swath of metrics and hope that one of them will lead them down a winning path. That approach is fraught with peril, as you will effectively still be guessing - given that there is no single point of truth, you will be spending time and effort on what might be a dead-end (e.g. measuring the completely wrong things).

Always ask yourself whether what you’re measuring is following the rule of three As.

Events are not metrics #

Another pitfall that is all too common is measuring events as if they were metrics. Someone ships a feature, they get really excited about it, and happily exclaim that with the release, there are seven million users who clicked a particular button on a given month! That’s exciting, right? Customers must love your feature! As I wrote a couple of years ago, usage is not engagement. Just because someone is clicking a button does not mean that it translates into useful product usage. Here is why - an action might be part of a larger user journey. A click on a button, for example - Checkout, means that the user might have an intent in mind that they want to put into action. That does not mean that they are going to go through with it.

If we just track how many people click the Checkout button, it’s easy to get pumped about it - wow, our customers are putting things into the cart, and checking out! How many of them actually went through the purchase flow, and paid you? How many dropped off and why? You can clearly see how the model by which you track events only falls apart. That is because events are not metrics - metrics are computed from events, usually more than one.

Some examples of good metrics #

We talked about some fundamentals, now let’s look at some examples of metrics that follow the rules above! For our example, we are going to come up with a hypothetical product, “Snackr”, a site that allows you to order snacks.

There are many more we can come up with; however this list should give you an idea of what you should be looking for when defining a metric. You can even re-use the same table format as above to be able to self-asses whether what you’re measuring is actually important.

Metrics are not good in isolation #

The last point worth mentioning is that whatever metrics that you’re tracking, when treated in isolation, they will never produce insights as impacftul, as those that are taken together. Knowing how many users you sign up without knowing the revenue impact is meaningless. Knowing how many sales you drive without understanding how many customers you’re losing every month is also meaningless. Treat all metrics as part of a bigger picture - the future of your product depends on them.